Skip to content

Conversation

@pcrespov
Copy link
Member

@pcrespov pcrespov commented Jun 5, 2025

What do these changes do?

Mitigates performance bottleneck when retrieving project(s)

Calls to get_services_for_user_in_product were causing significant delays, occasionally leading to timeouts when calling the following endpoints:

  • ♻️ GET /projects/{project_id}
  • ♻️ GET /projects

The root cause was a legacy call to a deprecated catalog REST endpoint that lists all services for a given user and product. This was originally used to determine whether the services associated with a project were accessible. However, more efficient alternatives are now available (e.g. catalog_service.batch_get_my_services), though the code was never updated.

As an interim solution, this PR introduces:

  • A TTL cache keyed by (user, product) to reduce repeated calls.
  • A smaller data footprint in the cached response to prevent unnecessary memory usage.

This change significantly improves the performance of the affected endpoints and mitigates the timeout issues reported in #7814.

Related issue/s

How to test

Dev-ops

@pcrespov pcrespov self-assigned this Jun 5, 2025
@pcrespov pcrespov added the release Preparation for pre-release/release label Jun 5, 2025
@pcrespov pcrespov added this to the Bazinga! milestone Jun 5, 2025
@codecov
Copy link

codecov bot commented Jun 5, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.54%. Comparing base (186af88) to head (fde55b4).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7836      +/-   ##
==========================================
- Coverage   86.70%   82.54%   -4.17%     
==========================================
  Files        1851      704    -1147     
  Lines       71968    33587   -38381     
  Branches     1218      176    -1042     
==========================================
- Hits        62403    27724   -34679     
+ Misses       9223     5805    -3418     
+ Partials      342       58     -284     
Flag Coverage Δ
integrationtests 64.25% <65.62%> (-0.03%) ⬇️
unittests 86.76% <100.00%> (+0.30%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 76.86% <ø> (-8.19%) ⬇️
agent ∅ <ø> (∅)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog ∅ <ø> (∅)
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 77.62% <ø> (-13.42%) ⬇️
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 88.34% <ø> (-1.74%) ⬇️
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage ∅ <ø> (∅)
webclient ∅ <ø> (∅)
webserver 83.73% <100.00%> (+0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 186af88...fde55b4. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pcrespov pcrespov force-pushed the is7814/fix-service-listing-in-project-read-operations branch from b8581d6 to 02172f7 Compare June 5, 2025 18:09
@pcrespov pcrespov changed the title 🐛 Fixes service-listing 🐛 Mitigates "Projects not being listed on osparc-staging.io" Jun 5, 2025
@pcrespov pcrespov marked this pull request as ready for review June 5, 2025 18:35
Copy link
Member

@odeimaiz odeimaiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great! thanks!

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@pcrespov pcrespov force-pushed the is7814/fix-service-listing-in-project-read-operations branch 2 times, most recently from bf20812 to 0012e8a Compare June 6, 2025 14:04
@pcrespov pcrespov force-pushed the is7814/fix-service-listing-in-project-read-operations branch from 0012e8a to f9904d3 Compare June 6, 2025 14:39
@pcrespov pcrespov enabled auto-merge (squash) June 6, 2025 14:48
@pcrespov
Copy link
Member Author

pcrespov commented Jun 6, 2025

@mergify queue

@mergify
Copy link
Contributor

mergify bot commented Jun 6, 2025

queue

🟠 Waiting for conditions to match

  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
      • #approved-reviews-by>=2
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved=0
      • -conflict
      • -draft
      • base=master
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=🤖-do-not-merge
      • label=🤖-automerge
      • any of: [🛡 GitHub branch protection]
        • check-skipped = deploy to dockerhub
        • check-neutral = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of: [🛡 GitHub branch protection]
        • check-success = system-tests
        • check-neutral = system-tests
        • check-skipped = system-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = unit-tests
        • check-neutral = unit-tests
        • check-skipped = unit-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = check OAS' are up to date
        • check-neutral = check OAS' are up to date
        • check-skipped = check OAS' are up to date
      • any of: [🛡 GitHub branch protection]
        • check-success = integration-tests
        • check-neutral = integration-tests
        • check-skipped = integration-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = build-test-images (frontend) / build-test-images
        • check-neutral = build-test-images (frontend) / build-test-images
        • check-skipped = build-test-images (frontend) / build-test-images

@pcrespov pcrespov added the 🤖-automerge marks PR as ready to be merged for Mergify label Jun 6, 2025
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jun 6, 2025

@pcrespov pcrespov requested a review from nereaiscamu June 6, 2025 14:49
@pcrespov pcrespov merged commit aaf1079 into ITISFoundation:master Jun 6, 2025
93 of 95 checks passed
@pcrespov pcrespov deleted the is7814/fix-service-listing-in-project-read-operations branch June 6, 2025 15:34
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Jun 6, 2025
92 tasks
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Aug 5, 2025
88 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖-automerge marks PR as ready to be merged for Mergify release Preparation for pre-release/release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Projects not being listed on osparc-staging.io

5 participants